Game theory of mind

This paper introduces a model of ‘theory of mind’, namely, how we represent the intentions and goals of others to optimise our mutual interactions. We draw on ideas from optimum control and game theory to provide a ‘game theory of mind’. First, we consider the representations of goals in terms of value functions that are prescribed by utility or rewards. Critically, the joint value functions and ensuing behaviour are optimised recursively, under the assumption that I represent your value function, your representation of mine, your representation of my representation of yours, and so on ad infinitum. However, if we assume that the degree of recursion is bounded, then players need to estimate the opponent's degree of recursion (i.e., sophistication) to respond optimally. This induces a problem of inferring the opponent's sophistication, given behavioural exchanges. We show it is possible to deduce whether players make inferences about each other and quantify their sophistication on the basis of choices in sequential games. This rests on comparing generative models of choices with, and without, inference. Model comparison is demonstrated using simulated and real data from a ‘stag-hunt’. Finally, we note that exactly the same sophisticated behaviour can be achieved by optimising the utility function itself (through prosocial utility), producing unsophisticated but apparently altruistic agents. This may be relevant ethologically in hierarchal game theory and coevolution

Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings

Recently, evidence has emerged that humans approach learning using Bayesian updating rather than (model-free) reinforcement algorithms in a six-arm restless bandit problem. Here, we investigate what this implies for human appreciation of uncertainty. In our task, a Bayesian learner distinguishes three equally salient levels of uncertainty. First, the Bayesian perceives irreducible uncertainty or risk: even knowing the payoff probabilities of a given arm, the outcome remains uncertain. Second, there is (parameter) estimation uncertainty or ambiguity: payoff probabilities are unknown and need to be estimated. Third, the outcome probabilities of the arms change: the sudden jumps are referred to as unexpected uncertainty. We document how the three levels of uncertainty evolved during the course of our experiment and how it affected the learning rate. We then zoom in on estimation uncertainty, which has been suggested to be a driving force in exploration, in spite of evidence of widespread aversion to ambiguity. Our data corroborate the latter. We discuss neural evidence that foreshadowed the ability of humans to distinguish between the three levels of uncertainty. Finally, we investigate the boundaries of human capacity to implement Bayesian learning. We repeat the experiment with different instructions, reflecting varying levels of structural uncertainty. Under this fourth notion of uncertainty, choices were no better explained by Bayesian updating than by (model-free) reinforcement learning. Exit questionnaires revealed that participants remained unaware of the presence of unexpected uncertainty and failed to acquire the right model with which to implement Bayesian updating

Functional MRI in Awake Unrestrained Dogs

Because of dogs' prolonged evolution with humans, many of the canine cognitive skills are thought to represent a selection of traits that make dogs particularly sensitive to human cues. But how does the dog mind actually work? To develop a methodology to answer this question, we trained two dogs to remain motionless for the duration required to collect quality fMRI images by using positive reinforcement without sedation or physical restraints. The task was designed to determine which brain circuits differentially respond to human hand signals denoting the presence or absence of a food reward. Head motion within trials was less than 1 mm. Consistent with prior reinforcement learning literature, we observed caudate activation in both dogs in response to the hand signal denoting reward versus no-reward

Production of β‑ionone by combined expression of carotenogenic and plant CCD1 genes in Saccharomyces cerevisiae

Background Apocarotenoids, like the C13-norisoprenoids, are natural compounds that contribute to the flavor and/or aroma of flowers and foods. They are produced in aromatic plantslike raspberries and rosesby the enzymatic cleavage of carotenes. Due to their pleasant aroma and flavour, apocarotenoids have high commercial value for the cosmetic and food industry, but currently their production is mainly assured by chemical synthesis. In the present study, a Saccharomyces cerevisiae strain that synthesizes the apocarotenoid -ionone was constructed by combining integrative vectors and high copy number episomal vectors, in an engineered strain that accumulates FPP. Results Integration of an extra copy of the geranylgeranyl diphosphate synthase gene (BTS1), together with the carotenogenic genes crtYB and crtI from the ascomycete Xanthophyllomyces dendrorhous, resulted in carotenoid producing cells. The additional integration of the carotenoid cleavage dioxygenase gene from the plant Petunia hybrida (PhCCD1) let to the production of low amounts of -ionone (0.073 ± 0.01 mg/g DCW) and changed the color of the strain from orange to yellow. The expression of the crtYB gene from a high copy number plasmid in this former strain increased -ionone concentration fivefold (0.34 ± 0.06 mg/g DCW). Additionally, the episomal expression of crtYB together with the PhCCD1 gene in the same vector resulted in a final 8.5-fold increase of -ionone concentration (0.63 ± 0.02 mg/g DCW). Batch fermentations with this strain resulted in a final specific concentration of 1 mg/g DCW at 50 h, which represents a 15-fold increase. Conclusions An efficient -ionone producing yeast platform was constructed by combining integrative and episomal constructs. By combined expression of the genes BTS1, the carotenogenic crtYB, crtI genes and the plant PhCCD1 genethe highest -ionone concentration reported to date by a cell factory was achieved. This microbial cell factory represents a starting point for flavor production by a sustainable and efficient process that could replace current methods.This work was funded by grants COPEC-UC 6C-063 and FONDECYT No 1130822, and the Novo Nordisk Foundation

Stimulus-Dependent Adjustment of Reward Prediction Error in the Midbrain

Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus–reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus

Novel Patient Cell-Based HTS Assay for Identification of Small Molecules for a Lysosomal Storage Disease

Small molecules have been identified as potential therapeutic agents for lysosomal storage diseases (LSDs), inherited metabolic disorders caused by defects in proteins that result in lysosome dysfunctional. Some small molecules function assisting the folding of mutant misfolded lysosomal enzymes that are otherwise degraded in ER-associated degradation. The ultimate result is the enhancement of the residual enzymatic activity of the deficient enzyme. Most of the high throughput screening (HTS) assays developed to identify these molecules are single-target biochemical assays. Here we describe a cell-based assay using patient cell lines to identify small molecules that enhance the residual arylsulfatase A (ASA) activity found in patients with metachromatic leukodystrophy (MLD), a progressive neurodegenerative LSD. In order to generate sufficient cell lines for a large scale HTS, primary cultured fibroblasts from MLD patients were transformed using SV40 large T antigen. These SV40 transformed (SV40t) cells showed to conserve biochemical characteristics of the primary cells. Using a specific colorimetric substrate para-nitrocatechol sulfate (pNCS), detectable ASA residual activity were observed in primary and SV40t fibroblasts from a MLD patient (ASA-I179S) cultured in multi-well plates. A robust fluorescence ASA assay was developed in high-density 1,536-well plates using the traditional colorimetric pNCS substrate, whose product (pNC) acts as “plate fluorescence quencher” in white solid-bottom plates. The quantitative cell-based HTS assay for ASA generated strong statistical parameters when tested against a diverse small molecule collection. This cell-based assay approach can be used for several other LSDs and genetic disorders, especially those that rely on colorimetric substrates which traditionally present low sensitivity for assay-miniaturization. In addition, the quantitative cell-based HTS assay here developed using patient cells creates an opportunity to identify therapeutic small molecules in a disease-cellular environment where potentially disrupted pathways are exposed and available as targets

Protein Disulfide Isomerase Regulates Endoplasmic Reticulum Stress and the Apoptotic Process during Prion Infection and PrP Mutant-Induced Cytotoxicity

<div><h3>Background</h3><p>Protein disulfide isomerase (PDI), is sorted to be enzymatic chaperone for reconstructing misfolded protein in endoplasmic reticulum lumen. Recently, PDI has been identified as a link between misfolded protein and neuron apoptosis. However, the potential for PDI to be involved in the pathogenesis of prion disease remains unknown. In this study, we propose that PDI may function as a pleiotropic regulator in the cytotoxicity induced by mutated prion proteins and in the pathogenesis of prion diseases.</p> <h3>Methodology/Principal Findings</h3><p>To elucidate potential alterations of PDI in prion diseases, the levels of PDI and relevant apoptotic executors in 263K infected hamsters brain tissues were evaluated with the use of Western blots. Abnormal upregulation of PDI, Grp78 and Grp58 was detected. Dynamic assays of PDI alteration identified that the upregulation of PDI started at the early stage and persistently increased till later stage. Obvious increases of PDI and Grp78 levels were also observed in cultured cells transiently expressing PrP mutants, PrP-KDEL or PrP-PG15, accompanied by significant cytotoxicities. Excessive expression of PDI partially eased ER stress and cell apoptosis caused by accumulation of PrP-KDEL, but had less effect on cytotoxicity induced by PrP-PG15. Knockdown of endogenous PDI significantly amended cytotoxicity of PrP-PG15, but had little influence on that of PrP-KDEL. A series of membrane potential assays found that apoptosis induced by misfolded PrP proteins could be regulated by PDI via mitochondrial dysfunction. Moreover, biotin-switch assays demonstrated active <em>S</em>-nitrosylted modifications of PDI (SNO-PDI) both in the brains of scrapie-infected rodents and in the cells with misfolded PrP proteins.</p> <h3>Conclusion/Significance</h3><p>Current data in this study highlight that PDI and its relevant executors may function as a pleiotropic regulator in the processes of different misfolded PrP proteins and at different stages during prion infection. SNO-PDI may feed as an accomplice for PDI apoptosis.</p> </div

Mapping Informative Clusters in a Hierarchial Framework of fMRI Multivariate Analysis

Pattern recognition methods have become increasingly popular in fMRI data analysis, which are powerful in discriminating between multi-voxel patterns of brain activities associated with different mental states. However, when they are used in functional brain mapping, the location of discriminative voxels varies significantly, raising difficulties in interpreting the locus of the effect. Here we proposed a hierarchical framework of multivariate approach that maps informative clusters rather than voxels to achieve reliable functional brain mapping without compromising the discriminative power. In particular, we first searched for local homogeneous clusters that consisted of voxels with similar response profiles. Then, a multi-voxel classifier was built for each cluster to extract discriminative information from the multi-voxel patterns. Finally, through multivariate ranking, outputs from the classifiers were served as a multi-cluster pattern to identify informative clusters by examining interactions among clusters. Results from both simulated and real fMRI data demonstrated that this hierarchical approach showed better performance in the robustness of functional brain mapping than traditional voxel-based multivariate methods. In addition, the mapped clusters were highly overlapped for two perceptually equivalent object categories, further confirming the validity of our approach. In short, the hierarchical framework of multivariate approach is suitable for both pattern classification and brain mapping in fMRI studies

A multidimensional evaluation framework for personal learning environments

Evaluating highly dynamic and heterogeneous Personal Learning Environments (PLEs) is extremely challenging. Components of PLEs are selected and configured by individual users based on their personal preferences, needs, and goals. Moreover, the systems usually evolve over time based on contextual opportunities and constraints. As such dynamic systems have no predefined configurations and user interfaces, traditional evaluation methods often fall short or are even inappropriate. Obviously, a host of factors influence the extent to which a PLE successfully supports a learner to achieve specific learning outcomes. We categorize such factors along four major dimensions: technological, organizational, psycho-pedagogical, and social. Each dimension is informed by relevant theoretical models (e.g., Information System Success Model, Community of Practice, self-regulated learning) and subsumes a set of metrics that can be assessed with a range of approaches. Among others, usability and user experience play an indispensable role in acceptance and diffusion of the innovative technologies exemplified by PLEs. Traditional quantitative and qualitative methods such as questionnaire and interview should be deployed alongside emergent ones such as learning analytics (e.g., context-aware metadata) and narrative-based methods. Crucial for maximal validity of the evaluation is the triangulation of empirical findings with multi-perspective (end-users, developers, and researchers), mixed-method (qualitative, quantitative) data sources. The framework utilizes a cyclic process to integrate findings across cases with a cross-case analysis in order to gain deeper insights into the intriguing questions of how and why PLEs work